Many of us have seen films containing remarkably realistic dinosaurs, aliens, animated toys and other fanciful creatures. Such animations are made possible by 3D computer graphics. A computer is used to model objects in three dimensions, and to display them on a screen such as your home television or computer screen. An artist can completely specify how each object will look as well as how it will change in appearance over time. The computer takes care of performing the many millions of tasks required to make sure that each part of the moving image is colored just right based on how far away it is, the direction in which light strikes each of the many objects in the scene, the surface texture of each object, and many other factors.
Because of the complexity of the 3D graphics generation process, just a few years ago computer-generated three-dimensional graphics was mostly limited to expensive specialized flight simulators, graphics workstations or supercomputers. The public saw the results of computer generated 3D graphics in movies and advertisements, but never actually interacted with the computers doing the 3D graphics generation. All that has changed with the availability of relatively inexpensive 3D graphics platforms such as the Nintendo 64® and various 3D graphics cards available for personal computers. It is now possible to produce exciting 3D animations and simulations interactively in real time on relatively inexpensive computer graphics systems in your home or office.
One goal of computer graphics is to provide the capability for a high degree of visual realism. This means that the computer ought to be able to model objects so they have visible characteristics just like real objects in the physical world. For example, to enable realistic lighting effects such as reflection, the computer should keep track of which objects have shiny surfaces and which objects have dull surfaces. Another important characteristic the computer should be able to model is how opaque or transparent an object is. The computer should allow you to see through transparent objects such as windows, but not through opaque objects such as stone walls.
Many computer graphics system model the opacity (transparency) of surfaces using a technique called “alpha blending.” Using this conventional technique, each image element is assigned an “alpha value” representing its degree of opacity. The colors of the image element are blended based on the alpha value—allowing one object to appear to be visible through another object. A further conventional technique called “alpha function” or “alpha test” can be used to discard an object fragment based on comparing the fragment's alpha value with a reference function or value. Alpha test may decide to not blend (i.e., to throw away) a potential part of an image because it is transparent and will therefore be invisible.
Alpha blending and alpha test are especially useful for modeling transparent objects such as water and glass. This same functionality can also be used with texture mapping to achieve a variety of effects. For example, the alpha test is frequently used to draw complicated geometry using texture maps on polygons—with the alpha component acting as a matte. By way of illustration, a tree can be drawn as a picture (texture) of a tree on a polygon. The tree parts of the texture image can have an alpha value of 1 (opaque), and the non-tree parts can have an alpha value of 0 (transparent). In this way, the “non-tree” parts of the polygons are mapped to invisible (transparent) portions of the texture map, while the “tree” portions of the polygon are mapped to visible (opaque) portions of the texture map.
The alpha component of a texture can be used in other ways—for example, to cut holes or trim surfaces. As one example, an image of a cutout or a trim region can be stored in a texture map. When mapping the texture to the polygon surface, alpha testing or blending can be used to cut the cutout or trimmed region out of the polygon's surface.
One interesting issue relates to the amount of alpha information that should be provided. In the real world, many objects are not completely transparent or completely opaque, but actually fall somewhere in between. For example, you can't see through etched glass, but you can see some light shine through it. Etched glass is an example of an object that is neither entirely transparent or entirely opaque, but is instead semi-transparent or “translucent.” Even objects we typically think of as being very transparent may not be entirely so but may instead be only semi-transparent. For example, pond water is relatively clear, but may have some cloudiness to it. You can see a certain distance through pond water, but it becomes increasingly opaque based on depth. Clouds, smoke and imaginary ghosts are other examples of semi-transparent objects you might want to model using a computer graphics system.
To model such semi-transparent objects, computer graphics systems in the past have used multi-bit alpha values that encode not just “opaque” and “transparent,” but also varying degrees of semi-transparency. However, additional memory is needed to store an alpha component for each image element. The amount of additional memory required depends on the size of the image (i.e., the number of image elements) and the amount of alpha information to be stored for each image element. Storing multi-bit alpha components for each of thousands of image elements can substantially increase the amount of memory required. Even in systems with lots of memory, it may be desirable for performance reasons (i.e., reduced memory access time) to minimize the amount of memory required to store a given image.
To avoid locking application developers to a particular set of memory requirements and/or memory access times, one approach used in the past was to make the image element encoding mode of the computer graphics system programmable. Under this approach, the programmer could select between different color encoding modes as dictated by the characteristics of the particular image being generated at the time. For example, some systems allowed the programmer to choose between single-word and double-word color encoding formats. The programmer could choose a single-word RGB format for images requiring lower color resolution and no transparency capabilities, or a double-word RGBA format for images requiring higher color resolution and transparency. Speed performance might suffer somewhat if the double-word format were selected (since two words instead of one need to be accessed for each image element), but this tradeoff might be worth it to enable more complex or interesting images to be generated.
While the approach of selecting between single-word RGB format and double-word RGBA format is very useful, it also has certain significant limitations. For example, in resource-constrained 3-D graphics systems such as 3-D home video games, it may be especially important as a practical matter to conserve memory usage and associated memory access time. This might mean, for example, that in the context of a real time interactive game, the programmer may rarely (if ever) have the luxury of activating the double-word RGBA mode because of memory space or speed performance considerations. In other words, even when using a system that provides an alpha mode, the game programmer may sometimes be unable to take advantage of it without degrading image complexity (e.g., number of textures) and/or speed performance.
One past proposed solution to this problem was to allocate a single bit of a single-word RGB color format for transparency. For example, if word length is 16 bits, five bits can be allocated to each of the three primary colors (red, green and blue)—and the extra bit could be used for transparency (alpha). While this approach is certainly useful in terms of very efficient use of available memory, it has the limitation of providing only a binary (on/off) alpha value (i.e., either transparent or opaque). This prior approach therefore cannot provide visual effects requiring more alpha resolution (semi-transparency).
By way of further explanation, along edges of cutouts, trim regions, and certain texture mapped images, it may be desirable to provide an alpha component value that lies somewhere between transparent and opaque. This capability can (coupled with conventional anti-aliasing techniques) smooth and soften transitions to increase realism. For example, in the real world, the edge(s) surrounding a cutout might not be an entirely sharp transition, but may instead have some smooth transition. Alpha blending based on a range of alpha components modeling semi-transparency coupled with anti-aliasing (which smoothes out the “jaggies” in a digitally stepped surface) can be used to effectively model natural edge rounding. But this technique requires the ability to model semi-transparency, and does not work well if the alpha component is limited to a single “on/off” value.
FIGS. 1(a) and 1(b) help to illustrate this. FIG. 1(a) shows an original texture of a tree on a black background. FIG. 1(b) shows this same texture with one bit of alpha resolution. A one-bit alpha texture appears unnatural around the edges. This is a problem when the texture appears in front of any other objects, or indeed when the background to the texture is anything other than plain black. The texture is pictured here on a plain white background.
We have realized that for many of the visual effects we wish to present in the context of video games and other 3D interactive applications, we want to be able to provide more than a single “on/off” (i.e., opaque or transparent) value, but we may not need a “full” resolution alpha component to accomplish our objectives. For example, to provide smooth anti-aliased edges on cutouts, we may not need full 8-bit alpha resolution to provide visually pleasing effects. Some type of reduced resolution alpha encoding for semi-transparency (e.g., two or three bits of alpha to encode transparent, opaque, and two or six intermediate semi-transparency values) may be sufficient.
FIG. 1(c) helps to illustrate this. FIG. 1(c) shows the same tree texture as FIGS. 1(a) and 1(b), but using a multiple bit alpha value. The texture is pictured here on a plain white background. A multiple-bit alpha texture appears far smoother around the edges, whatever is behind it.
The present invention takes advantage of this observation by providing, in one particular implementation, a compact image element encoding format that selectively allocates bits on an element-by-element basis to encode multi-bit alpha resolution. This technique may be advantageously used to allocate encoding bits within some image elements for modeling semi-transparency while using those same bits for other purposes (e.g., higher color resolution) in other image elements not requiring a semi-transparency value (e.g., for opaque image elements). Applications include but are not limited to texture mapping in a 3D computer graphics system such as a home video game system or a personal computer.
In accordance with one aspect of the invention, a stored data element format representing a portion of an image includes a multi-bit alpha component field that may or may not be present in a particular instance of said format. The format includes a further portion encoding at least one color component. This portion has a first length if said multi-bit alpha component field is present, and has a second length greater than said first length if said multi-bit alpha component field is not present.
In accordance with another aspect of the invention, a texture map includes a first texel encoded with a semi-transparency value and having first color resolution; and a second texel encoded without any semi-transparency value and having second color resolution greater than the first color resolution.
In accordance with a further aspect of the invention, a color image element encoding format comprises an indicator field indicating whether an instance of said format is capable of encoding semi-transparency. The format further includes at least one variable sized field encoding further information concerning the color image element. The at least one variable sized field has a first length if the indicator field indicates the format instance is incapable of encoding semi-transparency, and has a second length less than the first length if the indicator field indicates the format instance is capable of encoding semi-transparency.
In accordance with a further aspect of the invention, an image element encoding format includes a flag or other indicator that indicates whether the element has an associated a multi-bit alpha component. If the flag indicates that no alpha value is present, then the encoding format stores higher-resolution color information (e.g., five bits each of red, green and blue color information in one particular example). If, on the other hand, the indicator indicates that an alpha component is present, then the image element's color resolution is reduced (e.g., to four bits each of red, green and blue color information in one particular example), and the remaining bits are used to provide a multi-bit field to encode semi-transparency alpha information.
The present invention also provides a method of encoding an image element comprising specifying whether said image element will encode semi-transparency. If the specifying step specifies that said image element will encode semi-transparency, a set of plural bits within an encoding format is allocated to encode alpha. If the specifying step specifies that the image element will not encode semi-transparency, the set of plural bits is allocated to encode another characteristic of the image element (e.g., increased color resolution).
The present invention further provides an alpha component converter that converts between first and second resolutions of semi-transparency information, the converter quantizing or dequantizing first resolution semi-transparency information into a predetermined number of equal sized steps to form second resolution semi-transparency information.
The ability to vary the bit encoding format on an image-element-by-image-element basis provides the potential for enhanced image quality by, for example, increasing the color resolution of those image elements not needing an alpha component. Opaque image elements can use the bits that may otherwise be used for alpha encoding to achieve higher color resolution.
The variable bit field color encoding technique provided by the present invention is especially useful in encoding texture elements (texels) within a 3D graphics system. Such variable bit field color encoding can be used, for example, to provide a texture element multi-bit alpha component that allows smooth anti-aliased edges on cutouts and in other instances where semi-transparency encoding is useful, without requiring the programmer to invoke a double-precision color encoding mode for all image elements with resulting doubling of the total amount of storage space required. Furthermore, this technique can be used to preserve higher color resolution across most of an image while degrading it only locally in small image areas where semi-transparency is required. The loss of color resolution may not be noticeable in such small semi-transparent image areas.